Online Accelerated Gradient Descent and Variational Regret Bounds

In this note, we study Nesterov’s accelerated gradient descent method in an online setting and establish both variational static and dynamic regret bounds using the functional variation, which “match” previous regret bounds in terms of gradient variation. To the best of our knowledge, this is the first work to study Nesterov’s accelerated gradient method in an online setting and our regret bounds are better than previous variational regret bounds in terms of functional variation.